Earth Observation Scientific Workflows in a Distributed Computing Environment
Identifieur interne : 000598 ( Main/Exploration ); précédent : 000597; suivant : 000599Earth Observation Scientific Workflows in a Distributed Computing Environment
Auteurs : Terence L. Van Zyl [Afrique du Sud] ; Anwar Vahed [Afrique du Sud] ; Graeme Mcferren [Afrique du Sud] ; Derek Hohls [Afrique du Sud]Source :
- Transactions in GIS [ 1361-1682 ] ; 2012-04.
Abstract
Geospatially Enabled Scientific Workflows offer a promising toolset to help researchers in the earth observation domain with many aspects of the scientific process. One such aspect is that of access to distributed earth observation data and computing resources. Earth observation research often utilizes large datasets requiring extensive CPU and memory resources in their processing. These resource intensive processes can be chained; the sequence of processes (and their provenance) makes up a scientific workflow. Despite the exponential growth in capacity of desktop computers, their resources are often insufficient for the scientific workflow processing tasks at hand. By integrating distributed computing capabilities into a geospatially enabled scientific workflow environment, it is possible to provide researchers with a mechanism to overcome the limitations of the desktop computer. Most of the effort on extending scientific workflows with distributed computing capabilities has focused on the web services approach, as exemplified by the OGC's Web Processing Service and by GRID computing. The approach to leveraging distributed computing resources described in this article uses instead remote objects via RPyC and the dynamic properties of the Python programming language. The Vistrails environment has been extended to allow for geospatial processing through the EO4Vistrails package (http://code.google.com/p/eo4vistrails/). In order to allow these geospatial processes to be seamlessly executed on distributed resources such as cloud computing nodes, the Vistrails environment has been extended with both multi‐tasking capabilities and distributed processing capabilities. The multi‐tasking capabilities are required in order to allow Vistrails to run side‐by‐side processes, a capability it does not currently have. The distributed processing capabilities are achieved through the use of remote objects and mobile code through RPyC.
Url:
DOI: 10.1111/j.1467-9671.2012.01317.x
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 000521
- to stream Istex, to step Curation: 000521
- to stream Istex, to step Checkpoint: 000117
- to stream Main, to step Merge: 000599
- to stream Main, to step Curation: 000598
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Earth Observation Scientific Workflows in a Distributed Computing Environment</title><author><name sortKey="Van Zyl, Terence L" sort="Van Zyl, Terence L" uniqKey="Van Zyl T" first="Terence L" last="Van Zyl">Terence L. Van Zyl</name></author><author><name sortKey="Vahed, Anwar" sort="Vahed, Anwar" uniqKey="Vahed A" first="Anwar" last="Vahed">Anwar Vahed</name></author><author><name sortKey="Mcferren, Graeme" sort="Mcferren, Graeme" uniqKey="Mcferren G" first="Graeme" last="Mcferren">Graeme Mcferren</name></author><author><name sortKey="Hohls, Derek" sort="Hohls, Derek" uniqKey="Hohls D" first="Derek" last="Hohls">Derek Hohls</name></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:CA03DCD703518BA61B79176F012679B1EFD993F8</idno><date when="2012" year="2012">2012</date><idno type="doi">10.1111/j.1467-9671.2012.01317.x</idno><idno type="url">https://api.istex.fr/document/CA03DCD703518BA61B79176F012679B1EFD993F8/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">000521</idno><idno type="wicri:Area/Istex/Curation">000521</idno><idno type="wicri:Area/Istex/Checkpoint">000117</idno><idno type="wicri:doubleKey">1361-1682:2012:Van Zyl T:earth:observation:scientific</idno><idno type="wicri:Area/Main/Merge">000599</idno><idno type="wicri:Area/Main/Curation">000598</idno><idno type="wicri:Area/Main/Exploration">000598</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Earth Observation Scientific Workflows in a Distributed Computing Environment</title><author><name sortKey="Van Zyl, Terence L" sort="Van Zyl, Terence L" uniqKey="Van Zyl T" first="Terence L" last="Van Zyl">Terence L. Van Zyl</name><affiliation wicri:level="1"><country xml:lang="fr">Afrique du Sud</country><wicri:regionArea>Earth Observation Science and IT, Council for Scientific and Industrial Research</wicri:regionArea><wicri:noRegion>Council for Scientific and Industrial Research</wicri:noRegion></affiliation></author><author><name sortKey="Vahed, Anwar" sort="Vahed, Anwar" uniqKey="Vahed A" first="Anwar" last="Vahed">Anwar Vahed</name><affiliation wicri:level="1"><country xml:lang="fr">Afrique du Sud</country><wicri:regionArea>Earth Observation Science and IT, Council for Scientific and Industrial Research</wicri:regionArea><wicri:noRegion>Council for Scientific and Industrial Research</wicri:noRegion></affiliation></author><author><name sortKey="Mcferren, Graeme" sort="Mcferren, Graeme" uniqKey="Mcferren G" first="Graeme" last="Mcferren">Graeme Mcferren</name><affiliation wicri:level="1"><country xml:lang="fr">Afrique du Sud</country><wicri:regionArea>Earth Observation Science and IT, Council for Scientific and Industrial Research</wicri:regionArea><wicri:noRegion>Council for Scientific and Industrial Research</wicri:noRegion></affiliation></author><author><name sortKey="Hohls, Derek" sort="Hohls, Derek" uniqKey="Hohls D" first="Derek" last="Hohls">Derek Hohls</name><affiliation wicri:level="1"><country xml:lang="fr">Afrique du Sud</country><wicri:regionArea>Earth Observation Science and IT, Council for Scientific and Industrial Research</wicri:regionArea><wicri:noRegion>Council for Scientific and Industrial Research</wicri:noRegion></affiliation></author></analytic><monogr></monogr><series><title level="j">Transactions in GIS</title><idno type="ISSN">1361-1682</idno><idno type="eISSN">1467-9671</idno><imprint><publisher>Blackwell Publishing Ltd</publisher><pubPlace>Oxford, UK</pubPlace><date type="published" when="2012-04">2012-04</date><biblScope unit="volume">16</biblScope><biblScope unit="issue">2</biblScope><biblScope unit="page" from="233">233</biblScope><biblScope unit="page" to="248">248</biblScope></imprint><idno type="ISSN">1361-1682</idno></series><idno type="istex">CA03DCD703518BA61B79176F012679B1EFD993F8</idno><idno type="DOI">10.1111/j.1467-9671.2012.01317.x</idno><idno type="ArticleID">TGIS1317</idno></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">1361-1682</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Geospatially Enabled Scientific Workflows offer a promising toolset to help researchers in the earth observation domain with many aspects of the scientific process. One such aspect is that of access to distributed earth observation data and computing resources. Earth observation research often utilizes large datasets requiring extensive CPU and memory resources in their processing. These resource intensive processes can be chained; the sequence of processes (and their provenance) makes up a scientific workflow. Despite the exponential growth in capacity of desktop computers, their resources are often insufficient for the scientific workflow processing tasks at hand. By integrating distributed computing capabilities into a geospatially enabled scientific workflow environment, it is possible to provide researchers with a mechanism to overcome the limitations of the desktop computer. Most of the effort on extending scientific workflows with distributed computing capabilities has focused on the web services approach, as exemplified by the OGC's Web Processing Service and by GRID computing. The approach to leveraging distributed computing resources described in this article uses instead remote objects via RPyC and the dynamic properties of the Python programming language. The Vistrails environment has been extended to allow for geospatial processing through the EO4Vistrails package (http://code.google.com/p/eo4vistrails/). In order to allow these geospatial processes to be seamlessly executed on distributed resources such as cloud computing nodes, the Vistrails environment has been extended with both multi‐tasking capabilities and distributed processing capabilities. The multi‐tasking capabilities are required in order to allow Vistrails to run side‐by‐side processes, a capability it does not currently have. The distributed processing capabilities are achieved through the use of remote objects and mobile code through RPyC.</div></front></TEI><affiliations><list><country><li>Afrique du Sud</li></country></list><tree><country name="Afrique du Sud"><noRegion><name sortKey="Van Zyl, Terence L" sort="Van Zyl, Terence L" uniqKey="Van Zyl T" first="Terence L" last="Van Zyl">Terence L. Van Zyl</name></noRegion><name sortKey="Hohls, Derek" sort="Hohls, Derek" uniqKey="Hohls D" first="Derek" last="Hohls">Derek Hohls</name><name sortKey="Mcferren, Graeme" sort="Mcferren, Graeme" uniqKey="Mcferren G" first="Graeme" last="Mcferren">Graeme Mcferren</name><name sortKey="Vahed, Anwar" sort="Vahed, Anwar" uniqKey="Vahed A" first="Anwar" last="Vahed">Anwar Vahed</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/CyberinfraV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000598 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000598 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= CyberinfraV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= ISTEX:CA03DCD703518BA61B79176F012679B1EFD993F8
|texte= Earth Observation Scientific Workflows in a Distributed Computing Environment
}}
| This area was generated with Dilib version V0.6.25. |  |